(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 
International Bureau 

(43) International Publication Date 
22 March 2001 (22.03.2001) 




PCT 



(10) International Publication Number 

WO 01/20594 Al 



(51) International Patent Classification 7 : G10H 1/36 

(21) International Application Number: PCT/KROO/01042 

(22) International Filing Date: 

16 September 2000 (16.09.2000) 



(25) Filing Language: 

(26) Publication Language: 



Korean 
English 



(30) Priority Data: 

1999/39854 16 September 1999 (16.09.1999) KR 

(71) Applicant (for all designated States except US): 
HANSEULSOFT CO., LTD. [KR/KR]; Hyunjin Build- 
ing, 2nd floor, 798-30, Yeoksam-dong, Kangnam-gu, 
Seoul 135-080 (KR). 

(72) Inventor; and 

(75) Inventor/Applicant (for US only): KIM, Yun-Jong 
[KR/KR]; San 69, Namhyun-dong, Kwanak-gu, Seoul 
151-080 (KR). 



(74) Agent: SfflNSUNG PATENT FIRM; Haecheon Build- 
ing, 741-40, Yeoksam 1-dong, Kangnam-ku, Seoul 
135-081 (KR). 

(81) Designated States (national): AE, AG, AJU AM, AT, AU, 
AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CR, CU. CZ, 
DE, DK, DM, DZ, EE, ES, FI, GB, GD, GE, GH, GM, HR, 
HU, ID, IL, IN, IS, JP, KE, KG, KP, KZ, LC, LK, LR, LS, 
LT, LU, LV, MA, MD, MG, MK, MN, MW, MX, MZ, NO, 
NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, SL, TJ, TM, TR, 
TT, TZ, UA, UG, US, UZ, VN, YU, ZA, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZW), Eurasian 
patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), European 
patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, 
IT, LU, MC, NL, PT, SE), OAPI patent (BF, BJ, CF, CG, 
CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG). 

Published: 

— With international search report. 

For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



== (54) Title: METHOD AND APPARATUS FOR PLAYING MUSICAL INSTRUMENTS BASED ON A DIGITAL MUSIC FILE 



( START ) 



LOAD MIDI FILE 



I 



_j — 110 



CONVERTING TO MIDI FORMAT RLE 



Ir 120 



55 I SIMPLIFICATION OR COMPLICATION OF THE MUSICAL SCORE U 130 

^= GENERATING THE ELECTRONIC MUSICAL SCORE U HO 

= 7 

= | ADJUSTING COMPLEXITY ""[ -150 

1 INSERTING THE ADVERTISING INFORMATION k 160 

— T ; 

f-N I GENERATING THE VOS RLE | — ^70 

< ZLT 

(57) Abstract: The present invention relates to a system and method for playing musical instruments using a digital music file. The 
method for playing the musical instrument based on a computing system includes the steps of: generating a digital music file which 
^ stores a play order for each musical instrument and sound data for each musical instrument by partitioning the digital music file 
^ depending on the musical instruments; inputting music type data and musical instrument type data by a user; displaying an electronic 
score at a real-time wherein musical notes corresponding to each instrument to play the music are represented, selected from the 
music file; and playing the music according to the displayed electronic score by synthesizing a designated sound manipulated by the 
^ user and a background sound of the music wherein the designated sound is omitted. Accordingly, at least one user can play the music 
J^. based on the music file. At the same time, the user can play the music, selecting desired instruments and watching the displayed 
^ electronic score. 
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METHOD AND APPARATUS FOR PLAYING MUSICAL INSTRUMENTS 
BASED ON A DIGITAL MUSIC FILE 

Technical Field 

5 

The present invention relates to a method of converting 
a digital music file, in which tracks are classified on the basis 
of musical instruments to be played in such as a MIDI (musical 
Instrument Digital Interface), to a musical performance file 

10 (hereinafter, referred tc as a VOS (Virtual Orchestra System) 
file) which makes a direction of performance and playing back 
for each musical instrument, a musical performance system and 
method that one or more users select his own musical instrument 
and play the music through the electronic musical score displayed 

15 on a screen based on the VOS file, and a computer readable medium 
storing programs for realizing them. 

Background Art 

20 MIDI (Musical Instrument Digital Interface) is a kind of 

music file which can automatically play back sound or music 
through the electronic devices, and it has international 
standards for the compatibility of the data exchange among 
electronic musical instruments. Most electronic musical 

25 instruments can send/receive the data based on MIDI' s standards . 
For example, the electronic musical instruments for domestic use 
are divided into the electronic organ, the electronic piano, the 

1 
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electronic keyboard, etc. and these instruments, that synthesize 
and* output sound waves using the well-known analog or digital 
circuit, also follows the MIDI'S standards. 

In other words, MIDI has been considered as international 
5 standards for the compatibility in transmitting information 
about playing of musical instruments, and recommends the 
exclusive five pin-cable for sending/receiving channel and 
system messages . Also, MIDI recommends that the channel message 
may include the performance information, such as scale interval, 

10 dynamic and transformation of timbre programs, etc. 

Recently, there are many cases where the data can be shared 
with PCs (Personal Computers), and the electronic musical 
instruments can be controlled by the PCs. For the PCs to control 
the musical instruments with a MIDI port, the signals in the PC 

15 are required to be converted to MIDI signals for electronic 
musical instruments, and then the MIDI interface is necessarily 
required for this purpose. The MIDI interfaces may be divided 
into the intelligent type with CPU and'the non-intelligent type 
without it. In the present technologies, sixteen-dif f erent 

20 channel information can be transmitted through just one cable, 
therefore the MIDI file, which has the sounds of sixteen different 
musical instruments, can be easily transmitted (it is obvious 
that the extension of the channel information is available to 
the ordinary skilled in the art) . 

25 The present invention is suggested as a technique using 

these environments and resources. 

As a prior arts, "Beatmania" (it is the brandname) system 

2 
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of "KONAMI", which is well known as a music playing game system, 
will be explained concretely referring to Fig. 1. 

Fig. 1 is a block diagram illustrating a method for playing 
a music through the "Beatmania" system. 

As shown in Fig. 1, "Beatmania" of " KONAMI" is a music 
playing simulation game system, which directs a playing 
operation to users at a predetermined point of time and, when 
the users get into operation based on the direction, executes 
the manipulation such as a mixing of a preset playing sound wave 
(user input sound wave) and a effect sound wave (background music 
except the user input sound wave) given by Disc Jockey (DJ) . It 
directs the timing of playing operation to the user using an 
indicator, a kind of electronic musical score, and the user can 
play the music with five buttons and one turntable. According 
to the result of user's playing operation, "Beatmania" 
synthesizes and outputs the play sound wave and the effect sound 
wave . 

Although, this "Beatmania" system is useful in that it 
makes the user play the music easily by indicating the timing 
of playing operations, it requires a lot of efforts for 
professional musicians to compose the whole piece of music, 
because in the "Beatmania" system, the playing sound wave (user 
input wave) and the background sound wave should be separately 
designed for each musical instrument selected by the user. That 
is, in this system, the design of the play sound wave (user input 
wave) and the background sound wave should be repeatedly made 
as many as the number of musical instruments to be selected by 
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the user, because it is impossible to design the whole music at 
one time by partitioning the musical notes depending on each 
musical instrument . 

Also it has another problem in that once the music is made 
5 for "Beatmania", the change of the file requires the same complex 
procedures as making it first, because the musical performance 
file made through these procedures is in a type of a wave file. 

Even if the musical performance file is compressed, it 
needs a few or a few dozens of megabyte because' the file for the 
10 music playing game through these procedures is wave file, 
therefore the transmission of it needs large bandwidth. In 
result, in the current PC version, "Beatmania" has a technical 
limitation to a downgrade sound quality due to the reduction of 
an amount of data. 

15 

Disclosure of the Invention 

Therefore, to solve the above problems, it is an object 
of the present invention to provide a method of converting a music 

20 file to a VOS file by partitioning the musical notes for directing 
musical operation and playing sounds depending on each musical 
instrument in the computing system using digital music file (for 
example MIDI files), and a computer readable medium storing 
programs for implementing the above-mentioned method. 

25 It is another object of the present invention to provide 

a music playing system, which one or more users select their own 
musical instruments and play the music, showing electronic 
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musical scores displayed on the screen based on the VOS file; 
its method; and a computer readable medium storing programs for 
realizing them. 

. To achieve the first purpose of the invention, a method 
5 for converting a digital music file to a music playing file for 
each musical instrument comprises the steps of: classifying MIDI 
messages in MIDI files by type and storing them to memory; 
converting the delta time information of MIDI message to the 
absolute time information; and generating a VOS file by 

10 constructing a playing note message (for example, a user can 
select just one musical instrument through the whole playing time, 
change musical instruments during the playing, or plural users 
can play simultaneously with each their own musical instrument) . 
A method of converting file to the VOS file by classifying 

15 musical notes for playing each musical instrument comprises: 
classifying MIDI message and storing them to memory for obtaining 
necessary information from the message of MIDI file, in the 
apparatus with processor which converts a digital music file to 
a music playing file to be played for each musical instrument; 

20 carrying out the time conversion which change the delta time 
information to the absolute time information, and the 
classification of users based on relation to the musical 
instruments; generating a electronic score by simplifying the 
playing style to make it easy to play the musical instruments; 

25 generating a digital music file, which represents the notes in 
the timing of playing operation by partitioning the notes 
according to the playing order and musical instrument (track), 
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from the generated electronic musical score by changing the 
position of the notes, omitting some notes, adjusting the 
complexity of performance; and inserting additional 
information (advertising message, image, or sound clip for some 
5 corporations, goods, music) to said complexity coordinated 
musical score, and deciding the time to show the additional 
information to users. 

To accomplish said first purpose of the invention, a 
computer readable medium storing program code for realizing the 

10 present invention comprises: a first function of classifying 
MIDI message in MIDI files and storing them to memory; a second 
function of converting the delta time information of MIDI message 
to the absolute time information; and a third function of 
generating a VOS file by constructing a playing note message. 

15 Also, a computer readable medium storing program code for 

realizing the present invention comprises: 'a first function of 
classifying MIDI message and storing them to memory for obtaining 
necessary information from the message of MIDI file, in the 
apparatus with processor which converts a digital music file to 

20 a music playing file to be played for each musical instrument; 
a second function of carrying out the time conversion which change 
the delta time information to the absolute time information, and 
the classification of users based on relation to the musical 
instruments; a third function of generating a electronic score 

25 by simplifying the playing style to make it easy to play the 
musical instruments; a fourth function of generating a digital 
music file, which represents the notes in the timing of playing 
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operation by partitioning the notes according to the playing 
order and musical instrument (track), from the generated 
electronic musical score by changing the position of the notes, 
omitting some notes, adjusting the complexity of performance; 
5 and a fifth function of inserting additional information 
(advertising message, image, or sound clip for some corporations, 
goods, music) to said complexity coordinated musical score, and 
deciding the time to show the additional information to users. 
To achieve the second purpose, the music playing system of 

10 the present invention comprises: music file providing means for 
providing a music file to represent notes for each musical 
instrument at the timing of playing operation by classifying the 
notes depending on a playing order and a track corresponding to 
the musical instrument for said music; musical score display 

15 means for displaying each note for the musical instrument to be 
played as an electronic musical score; at least one playing 
operation input means for getting an playing operation input from 
the users according to the playing order of notes in the 
electronic musical score displayed by said musical score display 

20 means; musical processing means for storing a digital music file, 
converting the notes in a score to the electronic musical score 
representing the playing order, and providing the notes to the 
musical score display means, and outputting a designated sound 
manipulated by the playing operation input means and the 

25 background sound except the designated sound; music playing 
means for synthesizing and playing the designated sound and the 
background sound from the musical processing means; and output 
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means for outputting the music played by the music playing means. 

To achieve the second purpose, the music playing method of 
the present invention includes the steps of: generating music 
file representing the notes in the timing of playing order by 
5 partitioning the notes depending on the track of the playing order 
and musical instrument for the music; getting inputs of the title 
of music and the musical instrument to be played from users; 
. displaying electronic musical notes for musical instruments to 
be played as an electronic musical score that shows the playing 

10 order of the notes on a screen at a real-time; and synthesizing 
and playing the designated sound manipulated by users and the 
background sound except the designated sound according to the 
musical score displayed on the screen. 

Further, to achieve the second purpose of the present 

15 invention, a computer readable medium storing the program code 
for realizing the present invention includes the function of: 
generating music files for representing the notes for the music 
in the timing of playing order by partitioning the notes according 
to the playing order and musical instrument (track) in the music 

20 playing device with processor; getting inputs of the title of 
music- and the musical instrument to be played from users; 
displaying the musical notes for said music to be played as an 

• electronic musical score that shows the playing order of the notes 
on a screen at a real-time; and synthesizing and playing the 

25 designated sound manipulated by users and the background sound 

• except' the designated sound according to the musical score 
displayed on the screen. 
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Brief Description of the Drawings 

FIG. 1 is a block diagram for explaining a prior musical 
5 playing system of "Beatmania"; 

FIG. 2 is .a. flowchart showing a method of converting a 
: digital music file to a musical performance file for each musical 
instrument in accordance with the present invention; 

FIG. 3 is a comparative diagram of delta time information 
10 of a digital music file and absolute time information of a VOS 
file in accordance with the present invention; 

FIG. 4 is a flowchart illustrating a method of converting 
delta time information to absolute time information in 
accordance with the present invention; 
15 FIG. 5 is a schematic view illustrating a method of 

generating VOS electronic score in accordance with the present 
invention; 

FIG. 6 is a schematic view illustrating a method of 
adjusting the complexity during the generation of VOS electronic 
20 musical score; 

FIG. 7 is a flowchart illustrating an embodiment of a music 
playing system in accordance with the present invention; 

FIG. 8 is a schematic view illustrating a structure of the 
VOS file in accordance with the present invention; 
25 FIG. 9 is a schematic view illustrating a playing 

environment of VOS in accordance with the present invention; 

FIG. 10 is a schematic view illustrating a structure of 
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playing environment of VOS in accordance with an embodiment of 
the present invention; 

FIG . 11 is a flowchart illustrating a method of playing 
music in accordance with an embodiment of the present invention; 
5 . FIG. 12 is a schematic view illustrating a display of an 

electronic score in. a musical performance according to the 
present invention; 

FIG. 13 is a schematic view illustrating the output of 
designated sound in the musical performance according to the 
10 present invention; 

FIG. 14 is a schematic view showing an achievement of the 
. musical performance in accordance with an embodiment of the 
present invention; and 

FIG. 15 is a schematic view illustrating a playing 
15 • environment (karaoke) using in accordance with the present 
invention . 

Best mode for 'Carrying Out the Invention 

20 The present invention makes ordinary people, who is not 

skilled in musical instrument, play music easily using VOS format 
music files. Also, without a large space such as a studio with 
expensive musical instruments or special facilities, the present 
invention enables plural users connected through the computer 

25 network to constitute a virtual orchestra by selecting each their 
■ own musical instrument and then they play a music through a VOS 
electronic musical score displayed on each user's screen, using 

10 
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the VOS file format. 

For this purpose, as a tool for getting anyone to make VOS 
format music files easily, the present invention provides a 
method of converting digital music files, such as MIDI files, 
5 to the VOS files by classifying musical notes for directing 
musical operations on the basis of musical instruments and 
playing sounds for each musical instrument, and computer 
readable media storing the programs for realizing the same 
method. 

10 In the present invention, when MIDI file is previously 

provided for music, it may be used as a source file for the VOS 
file, otherwise, a MIDI file for the music should be made by a 
known- method and converted to the VOS file format (it means a 
specific file for representing the electronic score for each 

15 musical instrument for each user) ^so that the VOS electronic score 
for one or more musical instruments can be provided to the players, 
and the sound of selected musical instrument may be played by 
inputs according. to the users' operations. For the reference, 
MIDI file is already widely used in a musical entertainment such 

20 as noraebang (karaoke) because it is not only easy and cheap to 
make but also excluded from voice of a specific singer (so the 
royalty is not expensive) . 

The objects, features and utilities of the present 
invention will be clearer through the description and attached 

25 figures in the following. . Referring to the figures, the best 
mode in accordance with the present invention will be explained 
in detail. 

11 



WO 01/20594 



PCT/KROO/01042 



■ First, a method of generating VOS file by converting the 
MIDI file to the VOS format file will be explained. 

As illustrated in FIG . 2, to obtain necessary information 
.from messages in a MIDI format music file, the formation of the 
VOS file in accordance with the present invention classifies MIDI 
messages on the basis of type and stores them in a memory at step 
110. For the reference, the messages are divided into control 
messages (such as volume control , effect, etc), playing messages, 
and musical instrument configuration messages, etc. and a time 
domain of the entire message in the MIDI file is defined as a 
delta time. In the delta time system, all the time information 
of message are decided based on a time difference between the 
prior time information and the current time information (see FIG. 
3) . 

During the formation of the VOS file in the present invention, 
the MIDI format message defined as the delta time have to be 
converted to. those of the absolute time for a Virtual Orchestra 
playing . 

The meaning of the delta time and absolute time is easily 
understood by FIG. . 3 . illustrating the display of the time 
information for playing a message. That is, in FIG. 3, the left 
part shows -the delta time information and the right part the 
absolute time information. 

Also, the conversion of the delta time to the absolute time 
can be explained by the flowchart of FIG. 4. That is, the 
conversion of the delta time information of the MIDI format 
message to the absolute time information is accomplished by an 

12 
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initialization of setting the first value of the delta time as 
the initial value of the absolute time at step 121, and it 
repeatedly performs the steps of deciding if there is any message 
with converted delta time information at step 123 and setting 
5 the absolute time value of the current target message by adding 
the current delta time value-of target message to the previous 
absolute time value until there is no more unconverted delta time 
messages at step 125 . 

Then, in a construction of the note messages for users to 
10 play, the present invention forms a VOS file so that a single 
user may select just one-musical instrument through the whole 
playing time, change the instrument during the playing, or 
multiple users may play with each their own musical instrument 
simultaneously. 

15. . Although, in some cases, the playing operation input 

devices to- play all octaves without any modification such as 
simplification may be required, but inmost case ordinary people 
don't require to play a sophisticated music in such a way. The 
simplification of the playing operation is necessary so that the 

20 ordinary people play some musical instruments without excessive 
training, and reversely, for the specialized players, advanced 
playing operation is possible through the complication of the 
playing operation at step 130. . . 

For example, the simplification process of a musical score 

25 will be explained in the following referring to FIG. 5 and FIG. 
6. 

Referring to FIG. 5, a range of all the notes can be confined 

13 
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within just one octave by removing the octave element in notes 
("g" of 11a and lib in Fig. 5) . Among the notes, semi-tones can 
be replaced with whole-tones ("e" of 11a and lib in Fig. 5). 
Through these processes, the original musical score 11a can be 
simplified to a new type of electronic musical score lib at step 
140. However, such a new electronic musical score lib actually 
has both the original information for users to direct the normal 
playing operation and simplified information of the playing 
operation. 

Also, if necessary, additional modification, such as a 
change of the arrangement of the note and deletion of the notes 
to be played, can be made for the new style electronic score 
generated at step 150. For example, an editor can be used for 
adjusting the complexity by deleting the notes, as illustrated 
in FIG. 12. 

On the other hand, in case that the original score has an 
idle time, or the electronic score is modified by changing the 
arrangement or deleting the notes, additional advertising 
information (advertising words, image, sound clip for some 
corporation, goods, music) can be inserted for such an idle time. 
In this ca.se, the utility of VOS file is enlarged by specifying 
the display time with the advertising information at step 160. 

Finally, the VOS file is generated at step 170. 

In .Fig. 7 which illustrates a structure of the music playing 
system in accordance with the present invention, a reference 
numeral 2.1 represents an input device, 22 a screen display device, 
23 a VOS (Virtual Orchestra System) processing device, 24 a VOS 
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system file providing device, 25 a MIDI output device, 26 a 
speaker, respectively. 

As illustrated in FIG. 7, the music playing system in 
accordance with . the present invention includes the VOS file 
5 providing device 24 for supplying the VOS file, which represents 
the notes in the timing of playing order by classifying the notes 
according to playing order and musical instrument (track) for 
each music, the screen display device 22 for representing each 
note of music as an electronic musical score which shows the 

10 playing order of the notes, at least one input device 21 for 
getting the playing operation inputs from users according to the 
• playing order of the notes in the electronic musical score on 
.the. screen display device 22, the VOS processing device 23 for 
storing the VOS file, converting the notes for each musical 

15 instrument in the VOS file to the electronic musical score, which 
shows the playing order , providing it to the screen display device 
22 and outputting the designated sound manipulated by the input 
device 21 and the background sound, the MIDI output device 25 
for synthesizing and playing the designated sound and the 

.20 background sound from the VOS processing device 23, and the 
speaker 26 for outputting the music played by the MIDI output 
device 25. 

The input device 21 is a device for getting inputs from users, 
which are usually keyboards, mouses, MIDI instruments, joypads 
25 and simulative musical instruments (keyboard instruments, 
stringed instruments, a percussion instruments, wind 
instruments, etc) , and it may get the input from user operation 

15 
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.according to the playing order of notes in the electronic musical 
score on the screen display device 22. Here, a keyboard means 
a common input device used in PC system. Although a mouse doesn't 
have enough number of input keys as a main input device, it can 
5 be used as an assistant playing device with a keyboard or a joypad. 

The screen display device 22 is a screen for showing the 
VOS file to users as a type of electronic musical score, which 
is usually a monitor, a LCD display, or a TV. Here, the 
electronic musical score is a kind of score,' Which represents 
- 10 the playing order of the notes for music to be played in the VOS 
file. 

•The VOS processing device 23 has several functions of 
loading and storing the VOS file from the VOS file providing 
device 24 using a network operating system (for example, Windows 

15 NT, Windows 95/98), a set top bo#, a game device, a noraebang 
(karaoke) device etc. , converting the notes of VOS files for each 
musical instrument to the electronic musical score which shows 
the playing order, .providing the converted electronic musical 
score to the screen display device 22, getting and processing 

20 user inputs, outputting the MIDI messages (that is, the 
designated sound manipulated by the input device 21 and the 
background sound excluding the designated sound) to the MIDI 
output device 25. 

The VOS file providing device 24 is a device for providing 

25 the VOS file necessary for playing music using a network line, 
a hard disk, or a CD-ROM, etc. As stated in FIG. 2 to FIG. 6, 
the VOS file is generated by classifying the notes according to 
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the playing order and the musical instrument, and converting to 
VOS format using MIDI file. On the other hand, when there is 
no VOS file in a user terminal, it may be loaded from the external 
sources through the network, such as a remote host connected by 

..5 the wire or wireless internet, or a set top box using satellites. 

- The MIDI output device 25 synthesizes and plays the MIDI 
output messages (that is, the designated sound and the background 
sound) from the VOS processing device 23 using a MIDI module, 
a Software MIDI, and a FM synthesizer. 

10 Then, the virtual orchestra playing method will be 

concretely described in the following, which plural users can 
play each their own musical instrument in the network, such as 
internet or intranet, using the VOS file converted from MIDI file 
or some other sources for each musical instrument. 

15 in the preferred embodiment of the present invention, the 

. VOS file, which is classified depending on the playing order and 
the musical instruments for the music, is preferably classified 
into track messages for piano, violin, drum, trumpet, Korean gong, 
etc., as illustrated in FIG. 8. 

20 Here, the classification function (VOS file generating 

function) is to classify and store the ready made or created music 
on the basis of the musical instruments to be played with, editing 
and classifying the track from the MIDI file on the basis of 
musical instruments . 

25 . On the other hand, as shown in FIG 4, since the VOS file 

has data which are classified depending on tracks from ordinary 
music, the data is displayed on the screen as a type of electronic 



WO 01/20594 



PCT/KROO/01042 



score at a real-time. Therefore, when there is a user input from 
the input device according to the playing order in the electronic 
score, the playing function determines the note corresponding 
to the input at a real-time by the terminal loaded with the VOS 
5 and outputs the MIDI output message to the MIDI output device 
25. 

To accomplish a ensemble play function, in a single user 
playing mode, the user plays the designated sound of the selected 
musical instrument and the VOS system plays the sound of the rest 

10. of musical instruments (that is, background-sound) . On the other 
hand, in a multi-user playing mode, the user terminals with the 
VOS system may communicate with one another using communication 
protocol such as TCP/IP (Transmission Control Protocol/Internet 
Protocol) , IPX (Internetworking Packet Exchange) , and each user 

15 of the terminal with the VOS system may hear other sounds and 
plays his own musical instrument in accordance with it , therefore 
the virtual orchestra playing may be accomplished. 

The abstract playing environment as shown in FIG. 9 may be 
implemented by a playing environment as shown in FIG. 10. 

20 The VOS installed terminal may be a PC or a portable computer 

(notebook PC) with a operating system such as Windows or Linux. 

As an input device, a common keyboard, a keypad (that is, 
joypad with more than seven buttons as an option) , or a. well-known 
MIDI instrument may be employed. 

25 For the MIDI output device, a direct music (a software MIDI 

system for users without MIDI modules) or a hardware MIDI module 
producing a high quality sound with high quality musical 

18 
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instrument-sound fonts (such as the sound canvas, FM synthesizer, 
etc.) may be employed. 

In this real .VOS playing environment, the terminals 
communicate with one another using network protocols such as 
5 TCP/IP, IPX, etc. 

The VOS playing process in accordance with the present 
invention will be described in the following. 

Fig. 11 is a flowchart of an embodiment of the musical 
playing method in accordance with the present* invention which 
10 shows the VOS playing process using the VOS file. 

As illustrated in FIG. 11, in a first step of the VOS playing 
method. in accordance with the present invention, a list of music 
stored in the system is provided to the user, and then the music 
to be played is selected by the user through an input device at 
15 step 601. 

The VOS file generated through the above process is provided 
by the. VOS file providing device 24, and information for playing 
of musical instruments and advertisement are loaded from the VOS 
file at step 602, then the loaded information for playing and 
20 advertisement are'stored to a main memory device which has fast 
■ access time enough to use the- information at a real-time at step- 
603. 

Then, the musical instruments list for the selected music 
is provided to the user, and one of the musical instruments is 
25. selected by the user at step 604. 

The electronic score for the musical instrument played by 
• the user is displayed on the screen display device 22 at step 
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605. The electronic score may be scrolled according to the 
progress of playing in order to show the spot to be played, in 
case where the music is too long to show all the electronic score 
displayed on a screen. The progress of displaying the electronic 
score is illustrated in FIG. 12. That is, the notes of part A 
may be converted to those of B in the electronic score. 

Then, at step 606 the VCS processing device 23 recognizes 
the playing operation inputted by user, when the user plays with 
the input device 21 according to the electronic score on the 
screen display device 22. 

Then, for the musical instruments to be played by the user, 
the VOS processing device 23 outputs the notes corresponding to 
key inputs from the user at a real-time (that is, the MIDI output 
messages of the . designated sound) to the MIDI output device 25 
at step 607, and for the rest of the musical instruments to be 
played by the user, outputs the notes (that is, the MIDI output 
messages of the background sound) automatically to the MIDI 
output device 25 at step 608. For the electronic musical score 
without there being a simplification, for example, for playing 
with the real MIDI musical instruments, the inputs go out directly 
to. the user, without any simplification. For the simplified 
version of electronic musical score, the notes nearest to it may 
be a sent to the user as an output. 

For example, as illustrated in FIG. 13, if the user put downs 
'Do' after a lapse of 100s (C) from the starting point, and if 
'Do' of the 5th (b) octave exists at a position after a lapse 
90s, and that of 6th octave (a) should be at the 105s, the notes 
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nearer to the user input (a), that is, the note after a lapse 
of 105s may be decided to be played. 

Finally, the sound corresponding to the notes played by the 
user with a musical instrument and the background sound 
5 corresponding to the notes played by the VOS processing device 
23 automatically, not by the user, are mixed at a real-time by 
the MIDI output device 25, and go out through the speaker 26 at 
step 609. During this processing, the achievement or grade of 
the users' play, compared with the original music 9b, can be shown 

10 as illustrated in FIG . 14 at step 610. 

As stated in the above description, in the single player 
mode, displaying the electronic musical score at step 605, 
receiving the input from the user at step 606, outputting the 
designated sound of selected musical instrument at step 607, 

15 outputting. . the designated sound of unselected musical 
instruments automatically at step 608, making the ensemble 
process at step 609 are repeated until the end of the playing. 

On the other hsnd, in the double player mode, the ensemble 
is realized by mixing the designated sound of the selected musical 

20 instruments of the first and the second player with unselected 
musical instruments at a real-time and by making each player. 
.. listen to the other 1 s sound through the computer network. And, 
it is obvious that the ensemble with more than two players may 
be implemented in the same way. 

25 As. stated above, the present invention provides the 

ensemble play for the whole music by generating the VOS file from 
the MIDI file or other sources corresponding to it, which is 
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classified depending on the musical instruments and by making 
the users play each their musical instrument in the network 
. environment based on the VOS file. It means an overlap of the 
sound effect corresponding to the playing operation of the 
5 designated sound directed by the users and the background sound 
provided by the system. 

Since the present invention directs the timing of playing 
operation visually to the users thereof, not only the skilled 
player but also the ordinary people may play the music more easily, 

10 and the present invention may also adjust the complexity of 
playing by adding or deleting the input keys and make advanced 
sound effect such as a scratch effect. Also, in accordance with 
the present invention, the result is evaluated by comparing the 
playing operation directed to the users with the real operation 

15 by users, and is given to users as a feedback, so the present 
invention has usefulness not only for amusement but also for 
competition to get better grade. 

The present invention may be implemented by the single user 
mode or the multi-user mode, not only in the network but also a 

20 noraebang (karaoke), a game room, an arcade and a digital 
satellite broadcasting network. 

Referring to Fig. 15, in an application for noraebang 
(karaoke) , the VOS installed karaoke system is used for terminals, 
and the VOS controllers (preferably, simulative musical 

25 instruments such as keyboard instruments, stringed instruments, 
percussion instruments, wind instruments) for noraebang 
(karaoke) with ten or more buttons or other devices may be used 

22 



WO 01/20594 



PCT/KR00/01042 



for the input device. For output device, a well-known output 
system in karaoke may be used without change. The communication 
system may be achieved by the network system among the karaoke 
devices . 

On the other hand, in game rooms, by installing the VOS 
system in each terminal, a single user or plural users may play 
the music using each their own terminal. 

In arcades, by inserting coins, a selected music may be 
played with hands or feet, and the game may be continued according 
to the grade of game. In the digital satellite broadcasting 
system, the VOS player is installed on the set top box, the 
information on the selected music is transmitted from the digital 
satellite, and then the electronic musical score is displayed 
on the TV screen, therefore the music may be played according 
to the score. 

In this embodiment which one or more users play the music 
according to the electronic musical score displayed on the screen 
using- computer system, although the simplification of playing 
music is supposed so that one may play some musical instruments 
without much training, the wide range adjustment of complexity 
is possible by extending the musical range to two or five octaves, 
and it is obvious that such a transformation has the same effect 
as the embodiment of the present invention and is within the 
technical aspect of the present invention. Also, the MIDI 
instrument may be used for an input device because the MIDI 
instrument doesn ' t make real sound but send out signals for sound, 
and the correctness of the playing the MIDI instrument may be 
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compared with the real musical score. Further, feet as well as 
hands may be used as an input. Also, the simulative musical 
instruments used in a noraebang (karaoke) , a game room, an arcade 
can be made in the shape of real musical instruments for 
amusement . 

Industrial Applicability 

As apparent from the above description, according to the 
present invention, various musical instruments may be played 
more easily by forming the VOS file from the well-known MIDI file, 
and the ensemble is possible among a group of people making an 
advantage of chatting. And even unskilled people who are not 
trained for some musical instruments are able to play the music, 
and virtual orchestra may be implemented without any expensive 
real musical instruments, studio, or other equipments within the 
network environment such as internet or intranet. 

The present invention does not .confined to the said 
embodiments or . attached figures, and it is clear that the 
replacement,, transformation or modification is possible for the 
skilled man in the art within the technical aspect of the present 
invention . 
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Claims 

1. A music playing system supporting users to play music 
for a selected musical instrument based on the computing system, 
comprising; 

music file providing means for providing a music file to 
represent notes for each musical instrument at the timing of 
playing operation by classifying the notes depending on a playing 
order and a track corresponding to the musical instrument for 
said music- 
musical score display means for displaying each note for 
the musical instrument to be played as an electronic musical 
score; 

at least one playing operation input means for getting an 
playing operation input from the users according to the playing 
order of notes in the electronic musical score displayed by said 
musical score display means; 

musical processing means for storing a digital music file, 
converting the notes in a score to the electronic musical score 
representing the playing order, and providing the notes to the 
musical score display means, and outputting a designated sound 
manipulated by the playing operation input means and the 
background sound except the designated sound; 

music playing means for synthesizing and playing the 
designated sound and the background sound from the musical 
processing means; and 
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output means for outputting the music played by the music 
playing means. 

2 . The music playing system of Claim 1 , wherein the musical 
processing means has musical data classified depending on the 
track of the musical instrument and display the electronic 
musical score on the musical score display means at a real-time, 
wherein the musical processing means provides, in a single user 
mode, the designated sound of the user according to the electronic 
musical score and the background sound to the music playing means 
in the type of MIDI output message; and 

provides, in a multi user mode, the designated sound of the 
users according to the electronic -musical score and the 
background sound to the music playing means in the type of MIDI 
output message; 

3. The music playing system of Claim 1 or 2, wherein the 
music playing means make an ensemble play in a single user mode, 
by mixing the designated sound from the musical processing means 
with the background sound and playing the music and, in a multi 
user mode, by. mixing the designated sounds of the plural users 
from the musical processing means with the background sounds of 
the users. 



4. The music playing 
electronic musical score is a 
visually directs the playing 



system of Claim 3, wherein said 
digital electronic musical score, 
order by scrolling a part of the 
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musical score to be played according to playing time, and inputs 
adjusts a complexity. of the key for each musical instrument. 

5. The music playing system of Claim 4, wherein the 
electronic musical score is a digital electronic musical score, 
visually directs the playing order by scrolling the part of the 
musical score to be played according to playing time, a range 
of all the notes to one octave by deleting the octave element 
in notes to degrade the complexity of key input for each musical 
instrument, and changes all the semi-cones to whole-tones. 

6. The music playing system of Claim 4, wherein the 
electronic musical score is a digital electronic musical score, 
visually directs, the playing order by scrolling the part of the 
musical score to be played according to playing time, and the 
range of all the notes extends to two to five octaves by increasing 
the input keys to upgrade the complexity of key input for each 
musical instrument . 

7. The music playing system of Claim 4, wherein the at least 
one playing operation input means are at least one terminals 
providing the playing operation inputted by hands or feet of user 
according to the playing order of notes in said electronic musical 
score displayed on the musical score display means to the musical 
processing means. 

8. The music playing system of Claim 7, wherein the 
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computing system is for well known commercial network, a 
noraebang (karaoke), a game rooms, or the digital satellite 
broadcasting system. 

5 . 9. The music playing system of Claim 8, wherein the music 

includes ready made and created music. 

10. A music playing method in computing system, comprising 
. the steps of: 

10 a) generating music file representing the notes in the 

timing of playing order by partitioning the notes depending on 
the track of the playing order and musical instrument for the 
music; 

b) getting inputs of the title of music and the musical 
15 instrument to be played from users; 

c) displaying electronic musical notes for musical 
instruments to be played as an electronic musical score that shows 
the playing order of the notes on a screen at a real-time; and 

d) synthesizing and playing the designated sound 
20 manipulated by users and the background sound except the 

designated sound according to the musical score displayed on the 
screen . 

11, The music playing method of Claim 11, wherein, in 
25 single-user mode, the .step d) mixes the designated sound of the 

user according to the electronic musical score and the background 
sound, and play said music, in multi user mode, mixes the 
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designated sounds of plural users according to the musical score 
and the background sounds, and play the music. 

12. The music playing method of Claim 11, wherein the step 
5 a)- includes the steps of: 

al) classifying MIDI messages by type and storing them in 
memory to get the necessary information from musical message 
stored in the MIDI files; 

a2) converting the delta time information in the MIDI files 
10 to the absolute time information and classifying the users based 
on the relation to the musical instruments; 

a3) generating the electronic score by simplifying the 
playing style to make it easy to play the musical instruments; 
and 

15 a4) generating the music file by changing the position of 

notes, omitting notes needed, coordinating the complexity of 
playing, in the generated score through the step a3) . 

13 . The music playing method of Claim 12, further including 
20 the step of after the step a3) inserting the additional 
• . . information (advertising message, image, or sound clip for some 
corporations, goods, music) to the complexity coordinated 
musical score and deciding the time to show each additional 
information to users. 

25 

14. The music playing method of Claim 12, wherein the step 
a3) includes the step of deleting the octave element in notes 
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so that the range of all the notes can be confined to one octave, 
.and changing the semi-tones to the whole-tones. 

15. The music playing method of Claim 14, wherein the 
electronic musical score is a digital electronic musical score, 
visually directs the playing order by scrolling the part of the 
musical score to be played according to playing time, and a 
complexity of key inputs adjusts for each musical instrument. 

16. The music playing method of Claim 15, wherein the 
electronic musical score is a digital electronic musical score, 
visually directs the playing order by scrolling the part of said 
musical score to be played according to time, a range of all the 
notes confines to one octave by deleting the octave element in 
notes to degrade the complexity of key. input for each musical 
instrument, and all the semi-tones changed to whole-tones. 

■ ■ 1-7. The music playing method of Claim 15, wherein the 
electronic musical score is a digital electronic musical score, 
visually directs the playing order by scrolling the part of ' said 
musical score to be played according to-playing time, and the 
range of- all the notes extends to two or three octaves by 
increasing the input keys to upgrade the complexity of key input 
for each musical instrument. 

18-. The music playing method of Claim 15, wherein the 
computing system is for well known commercial network, a 
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noraebang (karaoke), a game room, or the digital satellite 
broadcasting system. 

19. The music playing system of Claim 18, wherein the music 
5 includes ready made and created music. 

20. A method for converting a digital music file to a music 
playing file for each musical instrument, comprising the steps 
of: 

10 a) classifying MIDI messages in MIDI files by type and 

storing them to memory; 

b) converting the delta time information of MIDI message 
to the absolute time information; and 

c) generating a VOS file by constructing a playing note 
15 message. 

21 . The method for converting a digital music file to a music 
playing file for each musical instrument of Claim 20, wherein 
the message includes control message, playing message, and 

20 musical instrument configuration message. 

22 . The method for converting a digital music file to a music 
playing file for each musical instrument of Claim 20, wherein 
the step b) includes the steps of: 

25 bl) generating a electronic score by simplifying the 

playing style to make it easy to play the musical instruments; 
and 
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b2) generating the music file, which represents the musical 
.notes to be played for each musical instrument according to the 
playing timing by classifying the notes depending on the playing 
order and musical instrument (track), by changing the position 
of notes, omitting notes needed, coordinating the complexity of 
playing in the generated electronic musical score through the 
step bl) . 

23 . The method for converting a digital music file to a music 
playing file for each musical instrument of Claim 22, wherein 
the step b) further includes the step of b3) inserting the 
additional information (advertising message, image, or sound 
clip for. some corporations, goods, music) to the complexity 
coordinated musical score and deciding the time to show each 
additional information to users. 

. 24 . In a device with a processor that converts digital music 
files to music playing files for each musical instrument, a 
computer readable medium storing instructions for realizing the 
functions of: 

a) . classifying MIDI message in MIDI files and storing them 
to memory; 

b) converting the delta time information of MIDI message 
to the absolute time information; and 

c) generating a VOS file by constructing a playing note 
message . 
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25. In., a device with a processor that partitions notes 
depending on musical instruments and stores them, a computer 
readable medium storing instructions for realizing the functions 
of: 

5 a) classifying MIDI message and storing them to memory for 

obtaining necessary information from the message of MIDI file, 
in the apparatus with processor which converts a digital music 
file to a music playing file to be played for each musical 
instrument ; 

10 b) carrying out the time conversion which change the delta 

time information to the absolute time information, and the 
classification of users based on relation to the musical 
instruments; 

c) generating a electronic score by simplifying the playing 
15 style to make it easy to play the musical instruments; and 

d) generating a digital music file, which represents the 
notes in the timing of playing operation by partitioning the 
notes according to the playing order and musical instrument 
(track) , from the generated electronic musical score by changing 

20 the position of the notes, omitting some notes, adjusting the 
complexity of performance. 

26. The computer readable medium of Claim 25, further 
comprising the function of e) inserting additional information 

25 (advertising message, image, or sound clip for some corporations, 
goods, music) to said complexity coordinated musical score, and 
deciding the time to show the additional information to users. 
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27. In a music playing device with processor, a computer 
readable medium storing instructions for realizing the function 
of: 

generating music files for representing the notes for the 
music in the timing of playing order by partitioning the notes 
according to the playing order and musical instrument (track) 
in the music playing device with .processor; 

getting inputs of the title of music and the musical 
instrument to be played from users; 

displaying the musical notes for said music to be played 
as an electronic musical score that shows the playing order of 
the notes on a screen at a real-time; and 

synthesizing and playing the designated sound manipulated 
by users and the background sound except the designated sound 
according to the musical score displayed on the screen. 
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